In
statistics
Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...
, the closed testing procedure is a general method for performing more than one
hypothesis test
A statistical hypothesis test is a method of statistical inference used to decide whether the data at hand sufficiently support a particular hypothesis.
Hypothesis testing allows us to make probabilistic statements about population parameters.
...
simultaneously.
The closed testing principle
Suppose there are ''k'' hypotheses ''H''
1,..., ''H''
''k'' to be tested and the overall type I error rate is α. The closed testing principle allows the rejection of any one of these elementary hypotheses, say ''H''
''i'', if all possible intersection hypotheses involving ''H''
''i'' can be rejected by using valid local level α tests; the adjusted p-value is the largest among those hypotheses. It controls the
family-wise error rate
In statistics, family-wise error rate (FWER) is the probability of making one or more false discoveries, or type I errors when performing multiple hypotheses tests.
Familywise and Experimentwise Error Rates
Tukey (1953) developed the concept of a ...
for all the ''k'' hypotheses at level α in the strong sense.
Example
Suppose there are three hypotheses ''H''
1,''H''
2, and ''H''
3 to be tested and the overall type I error rate is 0.05. Then ''H''
1 can be rejected at level α if ''H''
1 ∩ ''H''
2 ∩ ''H''
3, ''H''
1 ∩ ''H''
2, ''H''
1 ∩ ''H''
3 and ''H''
1 can all be rejected using valid tests with level α.
Special cases
The
Holm–Bonferroni method In statistics, the Holm–Bonferroni method, also called the Holm method or Bonferroni–Holm method, is used to counteract the problem of multiple comparisons. It is intended to control the family-wise error rate (FWER) and offers a simple test ...
is a special case of a closed test procedure for which each intersection null hypothesis is tested using the simple Bonferroni test. As such, it controls the
family-wise error rate
In statistics, family-wise error rate (FWER) is the probability of making one or more false discoveries, or type I errors when performing multiple hypotheses tests.
Familywise and Experimentwise Error Rates
Tukey (1953) developed the concept of a ...
for all the ''k'' hypotheses at level α in the strong sense.
Multiple test procedures developed using the graphical approach for constructing and illustrating multiple test procedures
are a subclass of closed testing procedures.
See also
*
Multiple comparisons
In statistics, the multiple comparisons, multiplicity or multiple testing problem occurs when one considers a set of statistical inferences simultaneously or infers a subset of parameters selected based on the observed values.
The more inferences ...
*
Holm–Bonferroni method In statistics, the Holm–Bonferroni method, also called the Holm method or Bonferroni–Holm method, is used to counteract the problem of multiple comparisons. It is intended to control the family-wise error rate (FWER) and offers a simple test ...
*
Bonferroni correction
In statistics, the Bonferroni correction is a method to counteract the multiple comparisons problem.
Background
The method is named for its use of the Bonferroni inequalities.
An extension of the method to confidence intervals was proposed by Oliv ...
References
{{Reflist
Statistical hypothesis testing
Statistical tests
Multiple comparisons